Deal with Conflicts

• Use the conflicts() function to figure out what conflicts you have
• Use package::fnName() to call a function directly without loading a package / to override conflicts
  • e.g., kableExtra should be loaded before tidyverse, but then tidyverse masks kableExtra::group_rows()

kable(tab) %>%
  kable_classic(html_font = "Times") %>%
  kableExtra::group_rows("Header", 1, 3)

Lesson 3

Start at the end

• What do you want your data to look like?
• What do they look like now?
• Now fill in the middle

Start at the end

HLM / MLM / MEM:
RE ex: time, trial, stimuli, group, study, day, w/in person conditions FE ex: gender, baseline age, b/w subject conditions, country, etc. (can also be an RE)

Start at the end

df %>% 
  pivot_longer(
    cols = RE_1_1:RE_2_2
    , names_to = c("RE1", "RE2")
    , names_sep = "_"
    , names_prefix = "RE_"
    , values_to = "DV"
  )

Note this is only possible / easy because of the naming scheme! If we had them named “RE1_1”, this would not have been possible / would have been CONSIDERABLY more difficult

Lesson 4

Don’t be afraid to split your data into chunks

• In alignment with starting at the end, a key strategy is knowing how you can chunk your data
• No right or wrong way to chunk, but some examples are:
  • items / values from the same scale / task (e.g., DV across trials / conditions)
  • baseline items (from other survey or from baseline wave)
  • outcome variables
  • descriptive variables
  • item-level variables v. composites

Lesson 5

Joining data requires a key, so be thoughtful and you’ll always be able to put the pieces together

• The most important thing when splitting data into chunks is to make sure you can put it back together
• This requires one (e.g., participant ID) or more (e.g., participant ID, wave) keys that allows R to match the right values together
• This should be the last thing you do
  • Please don’t create mega datasets where you tack things on to the raw data as you go
  • This will eat RAM and make your life harder (and sometimes could end up in you accidentally sharing identifying information!!)

Lesson 7

Establish a consistent naming scheme

• label objects relative to a stage or research question
  • e.g., nested_RQ1, RQ1_mods, raw_df
  • this will help you clear your environment of clutter
• use temporary objects repeatedly
  • e.g., if you need to use an object as an intermediary step, call it tmp and overwrite it as many times as is useful
  • You can always remove it using rm(tmp)

Lesson 8

You won’t remember details about raw variables or variables you create, document them

• Clearly document your raw data and planned transformations before (preregistration) or as (deviations or just reactive responses to aspects of the data) you clean your data
• Clearly document all new variables you create, including their scale, etc.

Reference data frames are great keys for ordering and renaming

out <- tribble(
  ~cat,     ~name,        ~scale,      ~long_name,              ~lab,   
"Outcome",  "dementia",   "0/1",       "Clinical Dementia",      "OR [CI]",
"Outcome",  "braak",      "1-5",       "Braak Stage",            "est. [CI]" 
)

tab %>%
  left_join(out %>% select(outcome = name, long_out = long_name, lab)) 

Lesson 10

Reorder everything using factors

• Often, there are specific orders we want / need strings to be in; this is where factors come in
• There’s a whole package for this called forcats.
• Reference data frames are a great way to order your variables

out <- tribble(
  ~cat,     ~name,        ~scale,      ~long_name,              ~lab,   
"Outcome",  "dementia",   "0/1",       "Clinical Dementia",      "OR [CI]",
"Outcome",  "braak",      "1-5",       "Braak Stage",            "est. [CI]" 
)

tab %>%
  left_join(out %>% select(outcome = name, long_out = long_name, lab)) %>%
  mutate(long_out = factor(long_out, levels = out$long_name))
# mutate(long_out = factor(oucome, levels = out$name, labels = out$long_name))

Reorder everything using factors

Remember this?

terms <- tribble(
  ~path,     ~new,                         ~level
  "i~1",     "Intercept",                  "Fixed",
  "s~1",     "Slope",                      "Fixed",
  "i~~i",    "Intercept Variance",         "Random",
  "s~~s",    "Slope Variance",             "Random",
  "i~~s",    "Intercept-Slope Covariance", "Random"
)

extract_fun <- function(m, trait){
  p <- parameterEstimates(m) %>%
    data.frame()
  # saveRDS(p, file = sprintf("results/summary/%s.RDS", trait))
  p %>%
    unite(path, lhs, op, rhs, sep = "") %>%
    filter(path %in% terms$path) %>%
    left_join(terms) %>%
    select(term = new, est, ci.lower, ci.upper, pvalue) %>%
    mutate(term = factor(term, levels = terms$new)) %>%
    arrange(term)
}

File structure and organization are your most important data cleaning & management tools

• Same thing goes for smaller objects
• Save those small ones, like summaries (e.g., from broom::tidy(), coef(), etc.), predicted values, random effects, etc. using the same file structure, and you always have everything at your fingertips
• Plus you can merge them more easily!
• This organization also transfers to GitHub for easy loading via raw links!

Lesson 13

Some things / functions are portable across projects, some need modification

• Some functions are portable:

z_scale <- function(x) (x - mean(x, na.rm = T))/sd(x, na.rm = T)
pomp_score <- function(x){
  rng <- range(x, na.rm = T)
  (x - rng[1])/(rng[2] - rng[1])*100
}

Some things / functions are portable across projects, some need modification

• Some are not:
• This function works for lavaan. With slight modifications, it could also work for broom::tidy() output

format_fun <- function(d){
  d %>%
    mutate(sig = ifelse(pvalue < .05, "sig", "ns")) %>%
    rowwise() %>%
    mutate_at(vars(est, ci.lower, ci.upper), round_fun) %>%
    mutate_at(vars(pvalue), pround_fun) %>%
    ungroup() %>%
    mutate(CI = sprintf("[%s,%s]", ci.lower, ci.upper)) %>%
    mutate_at(vars(est, CI, pvalue), ~ifelse(sig == "sig" & !is.na(sig), sprintf("<strong>%s</strong>", .), .)) 
}

Some things / functions are portable across projects, some need modification

• One possibility is to create an .R script that you can “source” (source("custom_functions.R"))
  • You could have general functions (e.g., z_scale(), pomp_score()) and use case specific ones (e.g., lavaan_format_fun() or broom_format_fun())
  • I often like to copy these into my R workflow because it means that everything is included in the scripts (even though the .R script can be included in the repo)

Lesson 14

Resources are finite: Be aware of how you use them

Resources are finite: Be aware of how you use them

Resources are finite: Be aware of how you use them

• Activity Monitor (Mac) or Process Monitor (Windows) is another great way to track general system usage across many programs, not just R
• I use this in particular when I’m doing parallelization
  • Sometimes threads stall (drop to 0% CPU or Memory)
  • Sometimes threads use way too much memory (and you start using swap)
  • It’s great to track this so you can interrupt

Lesson 15

Data frames are your friend

• They are the easiest objects to work with in R because of the number of dedicated tools and functions for working with them
• But they can get unwieldy (e.g., printing a data frame with hundreds of columns and thousands of rows)
• tibbles help with this but don’t always play nice
  • You can’t go from some classes to tibble directly
  • Instead go data.frame -> tibble

r <- cor(df, use = "pairwise")
r[upper.tri(r, diag = T)] <- NA 
r %>%
  data.frame() %>%
  as_tibble()

Lesson 16

RStudio / Pivot Cheat Sheets

Posit Cheatsheets

RStudio / Pivot Cheat Sheets

• Quarto
• RStudio
• RMarkdown
• lubridate
• stringr
• purrr
• readr
• tidyr
• dplyr
• ggplot2

Hacks

• The option / alt key and other shortcuts
• The tab key: “attempt completion”
• R templates
• GitHub Pages
• Functions without inputs